SYSTEM FOR TEXT ASSISTED TELEPHONY 



CROSS-REFERENCE TO RELATED APPLICATIONS 
[0001] This application is a continuation-in-part of Application No. 09/783,679, filed 

Febraary 14, 2001, pending, which is a continuation-in-part of Application No. 09/288,420, filed 
April 4, 1999, now U.S. Patent No. 6,233,314, which is a continuation of Application No. 
08/925,558 filed Sept 8, 1997, now U.S. Patent No. 5,909,482. 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR 

DEVELOPMENT 

[0002] Not applicable. 

BACKGROUND OF THE INVENTION 
[0003] The present invention relates to the general field of telephone communications. 

In more particular, the invention relates to systems to assist telephone communications by those 
persons who are deaf, hard of hearing, or otherwise have impaired hearing capability. 
[0004] Most modem human communications in both social and business environments 

takes place through sound communications. Yet within modem society there are many persons 
who have attenuated hearing capability. To assist those persons in making use of our telephonic 
communication system built for the hearing majority, there has been developed a system of 
telephone communication which has been principally used by the deaf community. That system 
makes use of a category of device known variously as a telecommunication device for the deaf 
(TDD), text telephone (TT) or teletype (TTY). Current TDDs are electronic devices consisting 
of a key board and a display as well as a specific type of modem, to acoustically or directly 
couple to the telephone line. Modem TDDs permit the user to type characters into their 
keyboard, with the character strings then encoded and transmitted over the telephone line to be 
displayed on the display of a communicating or remote TDD device. 

[0005] Most TDD communication is conducted in an idiosyncratic code specific to the 

community of TDD users. This code, known as Baudot, evolved historically at a time when 
many telecommunication devices for the deaf were based on mechanical or electromechanical 



devices rather than the current technology based on digital electronic components. Accordingly, 
the Baudot protocol was constructed for a set of constraints which are no longer relevant to 
present date devices. The original Baudot protocol was a unidirectional or simplex system of 
communication conducted at 45.5 Baud. The conventional Baudot character set was a character 
set consisting of 5 bit characters and the system encodes the bits of those characters in a two- 
tonal system based on carrier tones of 1400 and 1 800 Hertz. 

[0006] The system of TDD communications is widely used and in fact has become 

indispensable to the deaf commimity throughout the industrialized world. Deaf persons 
extensively communicate with their neighbors and with other deaf and hearing people remotely, 
using the TDD system. In addition, systems have been developed to facilitate the exchange of 
communication between the deaf community and hearing users who do not have access to or 
utilize a TDD device. In the United States, telephone companies have set up a service referred 
to as a "relay." A relay, as the term is used herein, refers to a system of voice to TDD 
communication in which an operator, referred to as a "call assistant, " serves as a human 
intermediary between a hearing user and a deaf person. Normally the call assistant wears a 
headset that communicates by voice with the hearing user and also has access to a TDD device 
which can communicate to the deaf user using a TDD appropriate protocol. In normal relay 
operations in the prior art, the call assistant types at a TDD keyboard the words which are voiced 
to her by the hearing user and then voices to the hearing user the words that the call assistant 
sees upon the display of his or her TDD. The call assistant serves, in essence, as an interpreting 
intermediary between the deaf person and the hearing person to translate from voice to digital 
electronic forms of communication. 

[0007] A system to assist users of the telephone system who are hard of hearing but not 

deaf has been described. This system, sometimes referred to as text enhanced telephony, makes 
use of the existence of relays to supplement telephone communications for users who can hear, 
but have attenuated hearing capabilities. This systems includes, in its simplest embodiment, a 
visually readable display connected in series between the telephone used by the assisted user and 
the connection to the telephone network. The text enhanced telephone call is connected through 
a relay which transmits both the voice of the hearing user at the other end and a text stream of 
the words spoken by that user on the same telephone line. The details of the concept and of 
some embodiments of that system are disclosed in U.S. Patent 6,075,842, the disclosure of 
which is hereby incorporated by reference. 
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BRIEF SUMMARY OF THE INVENTION 
[0008] The present invention is summarized in that a relay system to facilitate the 

translation of information and communication between deaf and hearing persons includes a call 
assistant who re-voices the words of the hearing person which are spoken to the call assistant. 
The words spoken by the call assistant are recognized by a speech recognition computer 
program which has been trained to the voice pattern of the call assistant, such that the words are 
promptly translated into a high speed digital communication protocol. That high speed digital 
communication message is then transmitted electronically promptly by telephone to a visual 
display accessible to the deaf person. 

[0009] It is an advantage of the invention described herein that the call assistant does not 

have to type most, if any, of the words spoken by the hearing person in the commimication 
session so that the overall speed of communications from the hearing person to the deaf person 
is dramatically increased. 

[00010] It is an object of the present invention that the design and utilization of a relay 

operated in accordance with the protocols described herein permits the introduction of small 
hand-held personal interpreter which will enable on the spot communications between deaf 
persons and hearing persons wherever the deaf persons might go. 

[00011] Other objects, advantages and features of the present invention will become 

apparent from the following specification when taken in conjimction with the accompanying 
drawings. 



BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS 
[00012] Fig. 1 is a schematic block diagram of a re-voicing relay. 

[00013] Fig. 2 is an exterior view of a personal interpreter enabled by the relay of Fig. 1 . 
[00014] Fig. 3 is a schematic block diagram of the personal interpreter of Fig. 2. 
[00015] Fig. 4 illustrates the operation of a captioned telephone service supported by a 
relay. 

[00016] Fig. 5 is a schematic illustration of the concept of the two wire captioned 
telephone device connection with sound inter-connection. 
[00017] Fig. 6 is a schematic illustration of the connection of a personal 

interpreter/captioned telephone device connected to two telephone lines. 
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DETAILED DESCRIPTION OF THE INVENTION 
[00018] The present invention is intended to provide an alternative arrangement for the 

delivery of text assisted telephone services, also called captioned telephone, to a telephone 
system user with diminished hearing. The present system is intended to take advantage of tM^o 
developments in the field of assisted telephone services, the personal interpreter and the re- 
voicing relay. The arrangement for using these capabilities here is designed not to provide text 
services for the profoundly deaf, but to provide a text assistance service for those who can hear, 
but who may have diminished hearing. This service is designed to supplement rather than 
replace the transmission of the spoken voice word to the assisted user. To assist imderstanding 
what is contemplated here, the basic technologies of the personal interpreter and the re-voicing 
relay will be described here first, after which the description will return to the main topic, text 
assisted telephone services or, as they are also known, captioned telephone services. 
[00019] The personal interpreter is intended to be a small portable device capable of 

delivering, with the support of a relay, text to a deaf user of any human conversation occurring 
in the presence of the personal interpreter. The personal interpreter is set up to connect 
telephonically to a relay, transmit spoken words to the relay, and then display for the assisted 
user the text of the words transmitted over the telephone connection to the relay. To make the 
personal interpreter work more seamlessly to produce more conversation-like communication, a 
faster relay methodology was needed. To fill this need, the re- voicing relay was designed. 
[00020] The re- voicing relay is based upon the underlying technology of using voice 

recognition software, operated by a call assistant (a "CA"), to assist in the voice to text 
translation inherent in providing relay services. The re-voicing strategy is described in U.S. 
Patent 5,909,482, the disclosure of which is hereby incorporated by reference. The re- voicing 
relay has its first implementation in providing voice to text transcription services for the deaf. 
However, the relay voice to text transcription service has use for users other than those who are 
deaf. It is envisioned that there are a number of hearing or partially hearing users who would 
have reason to benefit from voice to text transcription services. Relay voice to text service might 
also be useful for any application in which it is desired to supplement voice commimications by 
a text transcription of the voice spoken on the telephone- Thus, while the operation of the relay 
will sometimes be described here by referring to an "assisted user," who may be deaf or hard of 
hearing, but who also may be a normally hearing person who simply wants text assistance for 
some reason. The user at the other end of the line will be referred to here as the hearing user, 
simply for the purpose of having something to call that person, even though both users may be 



-4- 



hearing. 



Personal interpreter, 

[00021] Shown in Figure 2 is an illustration of what a personal interpreter 10 can look 

like. This would be a small hand held device typically the size of a small hardbound book. It 
would have a keyboard of minimal size, but useable by a deaf person who can type. It would 
have a two or four line display, but the display could be any size that conveniently fits in the 
case of the device. The device would also have a key or switch which would initiate its 
operation. 

[00022] Shown in Figure 2 is a schematic block diagram of the internal mechanics of the 
personal interpreter. The personal interpreter keyboard shown at 12 and its display as shown at 
14. Inside the interpreter itself is a microprocessor shown at 16. Not shown, but included 
within the personal interpreter, would be the appropriate memory and interface devices so as to 
allow the microprocessor to be programmed and to operate the personal interpreter and perform 
its functions, in a manner well known in the art. Also inside of the personal interpreter is a 
modem 1 8. The modem 1 8 is preferably a modem specifically designed for interface with the 
deaf telecommunications system. Most telecommunications with the deaf community are 
conducted using a Baudot type code. One useful alternative is for the modem be designed to 
use the enhanced form of Baudot communication known as "Turbo Code" (Ultratec), which is 
generally described in US Patents No. 5,432,837, No. 5,517,548, and 5,327,479, the disclosure 
of which is hereby incorporated by reference. Another alternative is that the modem use a new 
variant of Turbo Code, one which uses higher carrier frequencies (in the range of 3000-3500 
hertz) and a faster baud rate (over 100 baud). The most preferred alternative is for the modem to 
use a digital communication protocol which can both transmit and receive digital packets which 
contain either or both of voice and text. The output of the modem is preferably wired to a 
cellular telephone 20 included within the case of the personal interpreter 10. The cellular 
telephone 20 has a suitable antenna provided on it so that it may dial a cellular telephone 
network by radio frequency communications of the type normally conducted by cellular 
telephones. The personal interpreter also includes jack 28 to connect to a conventional wired or 
land-line telephone line as well. The personal interpreter also includes a microphone 22 and a 
speaker 24. A filter 26 connects the speaker 24 and the microphone 22 to the telephone 20. 
[00023] A brief description of the operation and functionality of the personal interpreter 

reveals the dramatic improvement and convenience and portability that this device gives to text 
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assisted people. A assisted user could go into an establishment, be it a government office or 
retail facility, in which there are only hearing persons. The assisted person would carry with 
him or her the personal interpreter 1 0. The assisted user would then place the personal 
interpreter 10 upon a counter or other surface, open it up, and press the initiation key or start 
button. The microprocessor 1 6 and modem 1 8 of the personal interpreter then power up and act 
in many ways like a normal TDD device operating in telecommunication standard, such as 
Turbo code. However, there is one critical difference. The start or initiation key further causes 
the microprocessor 16 of the personal interpreter to dial a relay to set up a relay commtmication 
session and includes in its communication with the relay a message, using the enhanced 
command features available in advanced teleconmiunication protocols, such as Turbo Code, to 
initiate a special format of relay call adapted for the personal interpreter. Other codes which 
permit command functions, such as ASCII or CC ITT, could also be used. The first operation is 
to activate the cellular telephone and direct the cellular telephone to dial the number of a relay 
operating in accordance with the method of the present invention. The cellular telephone dials 
the relay. Obviously, no wired connection is required to allow the cellular telephone function to 
establish a telephone connection with the remote relay, but alternatively the jack 28 to a 
conventional telephone line could be used. In addition, when the relay answers the telephone 
connection, the microprocessor 168 of the personal interpreter 10 is instructed to provide 
command codes to the remote relay. These conmiand codes, a feature possible through the use 
of Turbo Code, permits the personal interpreter to tell the relay that this is a personal interpreter- 
type relay commtmication session. All of this can happen in the time necessary to initiate the 
cellular call, perhaps two to ten seconds. 

[00024] Then, the assisted person can use the personal interpreter to translate words 

spoken by hearing persons in the presence of the personal interpreter into visually readable text. 
This is done by the personal interpreter 10 through an unseen relay. Words spoken by the 
hearing persons in the presence of the personal interpreter 10 are picked up by the microphone 
22. Those words are then transmitted through the cellular telephone 20 to the remote relay. The 
relay, operating as will be described below, then immediately transmits back, in enhanced Turbo 
Code, a digital communication stream translating the words that were just spoken. The words 
are received by the modem 18, and the microprocessor 16 in the personal interpreter 10, and it is 
displayed promptly upon the display screen 14. If the assisted person can speak, he or she may 
then emswer the hearing person with a spoken voice, or, the assisted person may alternatively 
type upon the keyboard 12. If the assisted user types on the keyboard 12, the personal 
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interpreter transmits the communication by digital communication to the relay. The call 
assistant at the relay then reads and speaks the words typed by the assisted user which are 
transmitted to the speaker 22 contained in the personal interpreter into a voice communication 
which can be understood by the hearing users. The filter 26 filters out the digital 
commimication frequencies from the sound generated by the speaker 22. Thus, in essence, the 
assisted person has a personal interpreter available to him or her at all times of the day or night 
wherever the assisted person is within the range of the cellular telephone system. Also, because 
the relay is preferably operating in accordance with the fast translation methodology described 
below, a very conversation-like feel can occur in the communication session between the 
assisted user and the hearing persons in the presence of the personal interpreter 10. In order for 
this commimication session to be satisfactory to the hearing users as well as the assisted person, 
however, the relay must operate exceedingly rapidly. It is, in part, to meet the need for the 
exceeding rapidity of this conversational style of conununication that the re-voicing relay 
protocol has been designed. 

Re-voicing relay. 

[00025] Shown in Figure 1 is a relay intended to provide the voice to text capability to 

support the personal interpreter, and which can also support other voice to text services to 
provide services for assisted users. Figure 1 is intended to show, in schematic fashion, how such 
a relay system can be set up. Shown at 32 is a telephone of a hearing person. Instead of a 
telephone of a hearing person, the input could also be the microphone of the personal interpreter 
translator 10 shown in Figures 2 and 3. The telephone of the hearing person 32 is coimected 
through a telephone line 34 to a voice input buffer 36 at the relay. The telephone line 34 can be 
an actual physical land line, or two pair between the telephones, or can be a cellular or other 
over-the-air telephone linkage. The voice input buffer 36 is a simple buffer to ensure temporary 
capture of the voice in the event that the call assistant gets behind and needs to buffer or delay 
the voice of the hearing person. In any event, the output of the input voice buffer 36 is provided 
to a headset 40 where earphones 38 produce the soimd of the remote speaking person in the ear 
of the call assistant. The call assistant is wearing the headset 40 and sitting at a computer 42 
capable of conmiunicating in an enhanced Baudot communication, such as Turbo Code or 
whatever other code protocol is being used. However, typically the call assistant does not type 
the words which the call assistant hears in his or her earphone 38. Instead, the call assistant then 
speaks the words which he or she hears in the earphones 38 into a microphone 39 in the headset 
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40. The microphone 39 on the headset 40 is connected to transmit the voice of the call assistant 
to the computer 42 at which the call assistant sits. 

[00026] The computer 42 has been provided with a voice recognition software package 

which can recognize the spoken voice of the call assistant and immediately translate words 
spoken in that voice into a digital text communication stream. It is a limitation of currently 
available speech recognition software that the software must be trained or adapted to a particular 
user, before it can accurately transcribe what words the user speaks. Accordingly, it is 
envisioned here that the call assistant operates at a computer terminal which contains a copy of a 
voice recognition software package which is specifically trained to the voice of that particular 
call assistant. It is also important that the voice recognition system be capable of transcribing 
the words of the voice of the call assistant at the speed of a normal human commimication. It 
has been foimd that a recently available commercial voice recognition package from Dragon 
Systems, known as '"Naturally Speaking," is a voice recognition software which will accomplish 
this objective and which will translate to digital text spoken words of a user at the normal speeds 
of human communication in conversation when operating on conventional modem personal 
computers. A voice recognition software system known as "Via Voice" from IBM provides 
similar ftmctionality. 

[00027] The computer terminal 42 of the call assistant then translates the text created by 

the voice recognition software to a modem 46 out through a telephone line 48 back to the 
display 50 located adjacent to the assisted person. The display 50 can be a conventional TDD 
located at the home of the remote assisted user, or can be the display 14 of the personal 
interpreter 10, or can be any other display or text capture device used by an assisted user. 
[00028] For reasons that will become apparent, there is also a connection from the 
microphone 39 of the headset 40 of the call assistant to the incoming telephone line 34 through a 
switch 52. The switch 52 can physically be an electrical switch located between the microphone 
39 and the telephone lines 34 and the computer 42 or, as an altemative, it can be a software 
switch operating in the computer 42 which passes the voice of the call assistant through to the 
telephone lines as voice, or not, under conditions which are selected by the call assistant, by 
choices he or she makes at the keyboard 44 of the computer 42. The switch 52 is functionally a 
single pole double throw switch although, of course, if this ftanction is performed by the 
computer it will be a logical not a physical switch. In the simplest embodiment, the switch 52 is 
a simple single pole dual throw foot switch readily accessible to the call assistant which passes 
the voice of the call assistant from the microphone either out onto the telephone line 34 or to the 
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computer 42. 

[00029] It is a further enhancement to the operation of the relay constructed in accordance 
with the present invention that the earphones 38 have noise attenuating capability. Noise 
canceling earphones are commercially available today or, for this purpose, the computer 42 can 
be provided with noise canceling sound generation software which would create sound 
transmitted to the earphone 38 so as to cancel the sounds of the call assistant's own voice. The 
noise attenuation or cancellation avoids distracting the call assistant, since he or she would then 
be less distracted by the words that he or she has spoken, and thus would be less likely to be 
distracted from the concentration of the task of re- voicing the sounds of the voice heard in the 
call assistant's ear. 

[00030] Similarly, another option which would be advantageous is that the software 
providing for the creation of the digital text string by voice recognition be buffered in its output 
flow to the modem 46. Before the computer 42 would pass the data on to the modem 46, the 
data would first be displayed on the computer screen of the computer 42 for review by the call 
assistant. The purpose of this option would be to permit the call assistant to use the keyboard to 
type or correct hard-to-spell words, or to create corrections of any misinterpretations created by 
the voice recognition software, from the words spoken by the call assistant. It is anticipated that 
if such an option is utilized, it would require fairly infrequent use of the keyboard by the call 
assistant, since frequent use would clearly slow down the through-put of the communications. 
[00031] The relay of Figure 1 can operate with normal TDDs or with a personal 

interpreter as shown in Figures 2 and 3, In either event, the hearing person speaks in the 
telephone 32 and the words are transmitted through the telephone line 34 to the voice buffer 36. 
The voice buffer 36, again operating under the control of the call assistant, would buffer the 
voice signals from the hearing user as needed for the call assistant to keep up. The call assistant 
would hear the voice of the hearing user through the earpiece 38 and then would re-voice those 
same words into the microphone 39. The words that the user speaks into the microphone 39 
would be fed to the computer 42 where the voice recognition software, trained to the voice of the 
call assistant, would translate those words into a digital text stream. The digital text stream 
would be turned into a digital communication stream by the modem 46 and passed on the 
telephone line 48 to a display 50 which can be observed by the assisted user. Experience has 
shown that using currently available technology the delay between the time the hearing user 
speaks into the telephone 32 and the time the words appear on the display 50 of the assisted user 
is a modest number of seconds. 
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[00032] In the reverse, when a non-speaking assisted user types onto his or her 
telecommunication device, the digital signals are transmitted to the computer 42 which displays 
them for the call assistant who then voices those words into the microphone 39 which words are 
then transmitted onto the telephone line 34. Note that the presence of the switch 52 is therefore 
important in this mode. Since the voice of the call assistant serves two different functions in the 
operation of this system, the signal on the call assistant's voice must be svdtched so that the 
hearing user 32 only hears the voice for the communications which are intended to be directed to 
that person. The switch 52 allows for the voice of the call assistant only to be directed to the 
hearing person at the appropriate times. 

[00033] Note that if the relay of Figure 1 is used to facilitate a translation based on a 
personal interpreter such as that shown in Figures 2 and 3, there will be only one telephone line 
between the personal interpreter and the call assistant. In essence, in a modification of Figure 1, 
the telephone 32 and the display 50 would both be within the personal interpreter 10. There 
would be only one telephone line, a cellular link, between the personal interpreter 10 and the call 
assistant. Note therefore that the voice of the call assistant and the digital communications 
created by the computer 42 would then travel on that same telephone linkage to and from the 
personal interpreter 10. It is therefore important for this embodiment that the personal 
interpreter 10 have appropriate filtering (i.e. the filter 26) to filter out the digital communication 
carrier frequencies of the digital communications protocol, so that they are not heard by hearing 
listeners in the presence of the personal interpreter 1 0. The telephone line must still carry voice 
signals, however, so that the spoken words articulated by the call assistant in response to digital 
instructions from the deaf user can be properly broadcast by the speaker contained within the 
personal interpreter. 

[00034] The provision for filtering of the digital frequencies can be done in any number 
of ways with two being the principal preferred methodologies. If Turbo Baudot 
commxmications are conducted at the conventional Baudot frequencies of 1400 and 1800 Hertz, 
the personal interpreter 10 could be provided with notch filters 26 to filter out signals at those 
particular fi-equencies. It has been found that such notch filters still permit the transmission of 
audible and understandable human speech, even if they filter at those particular frequencies. As 
an alternative, it is possible to change the Baudot frequencies to those which are much higher, 
such as frequencies of 3000 to 3500 Hertz. If this altemative is selected, the personal interpreter 
10 is then provided with a low pass filter which permits low frequency sounds to go to the 
speaker to be broadcast into the environment of the personal interpreter, while high frequencies 
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are excluded. 

[00035] It is also specifically envisioned that the filter of text communications signals 
from the voice signal can be done digitally or logically rather than by analog filter. For 
example, it is possible to implement the modem at site of the assisted user using a digital signal 
processor, or DSP, integrated circuit. Such a DSP chip can be programmed to recognize to 
separate voice signals from text communications signals and to pass along only the voice signals 
to the speaker at the location of the assisted user. It is also contemplated that the telephone 
linkage between the relay and the assisted user could be a digital connection rather than an 
analog telephone line. For example, the linkage between the relay and the assisted user could be 
a digital wireless linkage or an internet protocol linkage, wired or wireless. In either of these 
instances, the appropriate protocol calls for the transmission of packets of digital data, the 
packets being marked to indicate the type of information carried in the packets. The packets 
might contain, for example, voice signals which have been digitized or might contain simple 
digital information representing the text string of a conversation. In the instance of such a 
digital packet type communication protocol, the filtering out of the digital text information could 
be implemented simply by only converting the packets marked as voice back into sovmd. 
Packets marked as containing digital information would be used to recover the text information 
for display to the assisted user. 

Captioned telephone. 

[00036] As mentioned earlier, captioned telephone is a technology intended to provide 

text assistance to assisted user during the course of a voice telephone communication session. 
As will become apparent from the discussion below, a captioned telephone connection can be a 
single line connection or a multiple line connection. There are advantages and disadvantages of 
each approach. A single line connection is known in the art, as exemplified by U.S. Patent 
6,075,842, mentioned previously. The advantages and details of the use of a multiple line 
connection are described below. The discussion of captioned telephone will begin with a single 
line connection. 

[00037] Shown in Fig. 4 is an illustration of how a typical telephone call involving a 
single line captioned telephone would be set up. The hearing user at telephone 62 
communicates through a telephone line 64 with the relay, indicated at 66. The relay, a re- 
voicing relay, communicates through a telephone line 68 with the assisted user. At the site of 
the assisted user is a telephone 70 used by the assisted user and also a captioned telephone 
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device 72. The telephone 70 is conventional. The captioned telephone device 72 is constructed 
to accomplish two objectives. One objective is to filter, or separate, the digital signals carrying 
the text information from the voice signal. The other objective is to take the digital signals and 
create a visual display of the text information for the assisted user. This device is thus intended 
to assist the user to understand a greater portion of the conversation by providing a visually 
readable transcription of the text of the telephone conversation so that the assisted user can read 
any words that he or she cannot hear properly. While the captioned telephone device 72 of Fig. 
4 is illustrated as a separate stand-alone device from the telephone 70, those of skill in the art 
can readily appreciate that the two functions can be combined in a special capability telephone 
station. Such a special purpose station can both provide conventional telephone service and also 
include a display which can be used to provide captioning for an assisted user. 
[00038] The two line captioned telephone is indicated in Fig. 5. In its simplest 
embodiment, the two-line captioned telephone can be conceptually thought of as a personal 
interpreter used to provide text assistance to a telephone user. Thus in Fig. 5, the telephone of 
the hearing user 62 and the telephone of the assisted user 70 are connected directly by a simple 
single telephone connection, indicated by the telephone line 64. The assisted user then uses a 
personal interpreter/captioned telephone ("PICT") device 74 to connect to a relay 76 over a 
second telephone line or connection, indicated at 78. There must be some form of 
conununication between the telephone of the assisted user 70 and the PICT device 74, so that the 
voice of the hearing user can be transferred to the second telephone line 78 to the relay 76. At 
its simplest, the connection can be a simple acoustic connection, such as placing the sound pick- 
up microphone of the PICT device in such a location that the sound of the hearing person's 
voice is picked up and transmitted to the relay. A microphone connected to the PICT device 74 
could be placed on the handset of the assisted user' s telephone 70. However, it is preferred that 
the PICT device be electronically connected to the telephone of the assisted user so that the 
voice of the hearing user can be transmitted to the relay call assistant with minimal interference 
or noise. The PICT device includes a visually readable display and stiitable decoding electronics 
and software so that the device can receive and display a text stream received from the relay 
over the second telephone line. 

[00039] The main advantage of the two-line approach to captioned telephone is that the 

captioned telephone service can be added to a telephone call already in progress. By contrast, to 
use single line captioned telephone for an incoming call, the call must directed through the relay 
to the assisted user at the begiiming of the call. For a two-line captioned telephone call. 
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however, the call can be started as a normal telephone coironunication session, without the relay 
or the captioning. Then, if the assisted user decides captioning would be helpful to his or her 
understanding of the conversation, the captioning service can be added while the call is in 
progress. To add the captioning service, the assisted user simply has the PICT device dial the 
relay over the second telephone line. The voice of the hearing user is then transmitted over the 
second telephone line to the relay. The relay converts the voice to text and the text stream 
created by the relay returns to the assisted user, also over the second telephone line. Note that in 
this arrangement, as indicated in Fig. 5, voice and text are transmitted in opposite directions over 
a single telephone line between the PICT device and the relay, voice going to the relay and text 
returning on the same line. This arrangement of voice and text is the same as used for the 
personal interpreter. 

[00040] One device intended to implement the two-line captioned telephone approach is 
illustrated in Fig. 6. The PICT device 80 of Fig. 6 is an analog device connected in series 
between the telephone line 64 to the hearing user and the telephone 70 of the assisted user. As 
seen in Fig. 6, the telephone line 64 is connected directly through the PICT device 80 to the 
telephone 70. However, that telephone connection is also connected through a transformer 82 to 
a 2-wire to 4-wire converter 84. Connected to the output of the converter 84 is a low pass filter 
86, the output of which is the input to an amplifier 88. The output of the amplifier 88 is 
connected to a 2-wire to 4-wire converter 89 which connects, in turn, to the telephone line 78 
which connects to the relay (not shown here). The input signals firom the telephone line 78 are 
connected to a hi-pass filter 90, the output of which passes through a receive modem 92 to a 
microcontroller 94. The microcontroller operates the display 96. The microcontroller also 
includes the capability to generate DTMF dialing tones and impress those tones on the telephone 
line 78 when it is time to dial the relay. Thus here the single PICT device 80 connects to two 
telephone lines. A call can be initiated or received by the assisted user in the same manner as 
with a conventional telephone, using only the first telephone line 64. When the user wishes to 
invoke the captioning service, the assisted user presses a button,97, on the PICT device 80 that 
causes the device to automatically dial the relay on the second telephone line 78. The PICT 
device automatically transmits the voice signals on the first telephone line to the second 
telephone line (through the low-pass filter 86) for transmission to the relay. At the relay, the 
voice is converted into text and a text stream is sent back down the telephone connection 78 to 
the PICT device. The PICT device filters the text data, at the hi-pass filter 90, and transmits it to 
the microcontroller 94 for display to the user. None of the text data, and no digital carrier 
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signals, are heard by either the hearing user or the assisted user. Normally the hearing user does 
not even need to be aware that captioning is being used on the call. 

[00041] This example assumes that the digital carrier signals for the text message are 

carried on high frequencies, and the low pass and high pass filters are used to separate voice 
from text. While this filtering can be implemented as an analog filter, in many telephonic 
systems today, digital forms of communication are used. Using a digital communication 
protocol between the relay and the PICT device, commimication is in the form of digital data 
packets of either text or digitized voice. In that event, the logical filtering simply consists of not 
creating any acoustic noise from the packets designated as carrying text. 
[00042] Several physical versions of the PICT device are contemplated. In the version 
illustrated in Fig. 5, the PICT device is a stand-alone appliance inserted in series between the 
telephone of the assisted user and the telephone jack on the wall of the user's home. It may also 
be convenient for the user to package the PICT device as a single two-line imit intended to 
provide both conventional telephone service, as well as captioned telephone services. The 
device could look like a normal telephone, but have a display. The captioned telephone feature 
would be activated by a button or key, which would cause the unit to dial a pre-selected relay 
and set up a captioning session, whether or not a call on the other line is yet in progress. The 
voice signals incoming on the first telephone line from the hearing user would be transferred 
electronically to the telephone line to the relay. Text signals would be transmitted on that same 
telephone line back from the relay to the PICT device, which would display that text for the 
assisted user. Since the re-voicing relay operates with only a very slight time delay, the text 
display for the assisted user would trail the hearing person's voice slightly, but the text would 
still assist the user in comprehending the conversation. 

[00043] It is a desirable feature of the two-line captioned telephone arrangement that the 
use of the captioning service is transparent to the hearing user. The hearing user would dial to 
connect to the assisted user as with any other telephone user. The assisted user invokes the 
captioned telephone service without the need to involve the hearing user at all. The service can 
be used equally well and transparently for both incoming and outgoing calls. 
[00044] It is also envisioned that the captioned telephone service can be implemented in a 

way that uses three telephone lines. One line is for voice communications with the hearing user. 
The second line is directly connected to the first line so that the voice of the hearing user is 
transmitted to the relay. The third line is a connection to transmit the text stream from the relay 
to the station of the assisted user. 
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[00045] In the implementation of a relay providing captioned telephone service, the relay 
receives the voice of the hearing user and transmits both a digital text message stream and the 
voice of the hearing user over a telephone connection to the station of the assisted user. It is to 
be understood, however, that a conventional telephone single line connection is only one 
example of a telephonic connection that can be used in this arrangement. Digital wireless 
connection, or PCS connection, or even internet protocol (IP), wired or wireless connection can 
be used to connect the relay to the assisted user, so long as the connection in capable of 
transmitting voice to that user. 

[00046] Thus the term telephone line as used in this specification is intended not only to 
apply to a traditional land-line two-wire telephone line, but also to all equivalents that offer 
similar functionality. Each of the telephone lines could be, for example, a portion of the 
bandwidth of an ISDN or DSL service. The telephone line could be an analog or digital cellular 
telephone link or a PCS connection. The PICT device could also be connected to the internet 
communication in IP, and in that event the two telephone lines would simply be simultaneous 
digital data exchange with two remote locations. 

[00047] It is to be understood that the present invention is not limited to the particular 

illustrations and embodiments disclosed above, but embraces all such modified forms thereof as 
come within the scope of the following claims. 
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